Documentation update for 1.19 #597
Conversation
Fixing the following flags setting: cutlass_fp8_supported, use_marlin to False for HPU
Reverts HabanaAI#551 Different formatting
Revert changes in lm eval test
README_GAUDI.md
Outdated
|
|
||
| #### 1. Build and Install the stable version | ||
|
|
||
| Periodically, we are releasing vLLM to allign with Intel® Gaudi® software releases. The stable version is released with a tagg, and supports fully validated features and performance optimizations in Gaudi's [vLLM-fork](https://github.com/HabanaAI/vllm-fork). To install the stable release from [HabanaAI/vLLM-fork](https://github.com/HabanaAI/vllm-fork), run the following: |
README_GAUDI.md
Outdated
| ```{.console} | ||
| $ git clone https://github.com/HabanaAI/vllm-fork.git | ||
| $ cd vllm-fork | ||
| $ git checkout v1.19.0 |
There was a problem hiding this comment.
Probably will need to replace with proper tag like in v0.5.3.post1+Gaudi-1.18.0
There was a problem hiding this comment.
@bartekkuncer please verify if that makes sense.
There was a problem hiding this comment.
@michalkuligowski makes a good point especially that in release note we provide instruction with the use of a tag, so this change will make these two consistent.
| $ cd vllm | ||
| $ git clone https://github.com/HabanaAI/vllm-fork.git | ||
| $ cd vllm-fork | ||
| $ git checkout v1.19.0 |
There was a problem hiding this comment.
Probably will need to replace with proper tag like in v0.5.3.post1+Gaudi-1.18.0
There was a problem hiding this comment.
@bartekkuncer please verify if that makes sense.
There was a problem hiding this comment.
@michalkuligowski makes a good point especially that in release note we provide instruction with the use of a tag, so this change will make these two consistent.
| @@ -11,7 +11,7 @@ Please follow the instructions provided in the [Gaudi Installation Guide](https: | |||
| - OS: Ubuntu 22.04 LTS | |||
| - Python: 3.10 | |||
| - Intel Gaudi accelerator | |||
| - Intel Gaudi software version 1.18.0 | |||
| - Intel Gaudi software version 1.19.0 | |||
|
|
|||
| ## Quick start using Dockerfile | |||
| ``` | |||
There was a problem hiding this comment.
Or explain what you have in mind so someone else can do it?
There was a problem hiding this comment.
Other vendors seem to have it in the similar way as here:
https://docs.vllm.ai/en/latest/getting_started/openvino-installation.html#quick-start-using-dockerfile
https://docs.vllm.ai/en/latest/getting_started/cpu-installation.html#quick-start-using-dockerfile
https://docs.vllm.ai/en/latest/getting_started/arm-installation.html#quick-start-with-dockerfile
https://docs.vllm.ai/en/latest/getting_started/xpu-installation.html#quick-start-using-dockerfile
There was a problem hiding this comment.
Please see how document is structured:
- Quick start using Dockerfile
- Build from source
2.1 Environment verification
2.2 Run Docker Image
2.3 Build and Install vLLM
Questions:
- is (1.) full alternative to (2.)? If so, I would add one liner as
- "You can quickly set up vLLM using latest Intel Gaudi docker and vllm verson "
- is (2.1 Env verification) common to (1.) and (2.)?
Co-authored-by: Piotr Bocian <[email protected]>
bartekkuncer
requested review from
kzawora-intel and
madamczykhabana
as code owners
Merges #507 and #597, updates changelog and adds minor changes. --------- Co-authored-by: Bartosz Kuncer <[email protected]>
No description provided.